ITF is found in SCIENCE. MAINTENANCE IN PROGRESS

At 6:01 UTC I set the ITF in Maintenance Mode, here is the list of the activities communicated to the Control Room:

• Cleaning of Central Building, Laser Lab (external firm managed by Gargiulo);
• VAC Refill (Vacuum Team);
• 6:30 UTC Refill TCS chillers and change the filter of the aux cooling system (N. Menzione and J. Gargiulo)
• Operator Periodic Tasks (see section below);
• check of magnetometers at CEB (I. Fiori, M, Graziaplena)
• F. Sorrentino DET SQZ box alignment 
• D. Soldani checks AHU DET,INJ, CEB 
• 08:34-08:40 UTC replacement of WI etalon power supply (R. Cavalieri, V. Dattilo)
• 09:10 UTC FSR scan with 4 times the normal iterations value
• DMS flags added to horizontal positions SBE benches (H. J. Bulten)
• upgrades on the underlaying virtual infrastructure where the fileserver are hosted (E. Kraja, S. Cortese)

Periodic Tasks carried out by the Operator:

• From 6:19 UTC to 6:36 UTC, Check of IMC Working Point and Position;
• 07:04-07:09 UTC OMC locked in SINGLE_BOUNCE_NI
• From 7:11 UTC to 7:32 UTC - OMC Scan in Single Bounce configuration;
• At 8:00 UTC - TCS Thermal Camera Reference;
• At around 06:23 - 06:35 UTC - TCS Power Checks (see table below for results):

ITF found in LOW_NOISE_3_SQZ and in Science Mode. It kept the lock for the whole shift.

Guard Tour (UTC):
21:01 - 21:29
23:02 - 23:29
01:01 - 01:29
03:01 - 03:29

TCS
At around 4:25 UTC the value of TCS_WI_CO2_PWROUT suddenly decreased (see attached plot). Expert informed via mail of the issue.

I found ITF in SCIENCE MODE; It remained locked for the whole shift. 

No DMS Events to report

Guard Tour (UTC): 
17:08 - 17:35 
19:01 - 19:29

The automated noise injections on Friday 26th worked as intended: there were two minutes of noise injection on each longitudinal DoFs (Figs.1-5).

The coherence with Hrec is unsatisfying though (Figs.6-10). A finer tuning of the amplitudes of the noise injections might be necessary.

I found ITF in SCIENCE; It remained locked for the whole shift.

I'm repeating the analysis described in #63996 for the second week of O4b. The selected data segment is: 2024-04-20 22:14:40 to 2024-04-21 18:30:12 (72932.0 sec).

As before, I attach the CSV with the identified frequencies and the PDF with the various regions of the spectrum.

In 2017 the PRC was measured to have a wrong length by 3-4mm: https://logbook.virgo-gw.eu/virgo/?r=36884

I have checked in Optickle that mistuning the sideband frequency by a 2 kHz (which should be equivalent to a 4mm cavity length change), the sideband behavior becomes asymetric with PRCL offset. WIth the sideband power increasing by 1-2% in one direction and decreasing by 1-2% in the other direction, when changing the PRCL offset by ~0.2nm. This is similar to the measurements done last Friday, but it disagrees with the PRCL calibration done a year ago: https://logbook.virgo-gw.eu/virgo/?r=36884. That calibration would say that the PRCL offset was changed by 1.5nm, which would make the sideband power change by a few tens of percent. Maybe that calibration is no longer valid, as some normalizing factors could have been changed when chaning PRCL error signal from 8MHz to 6MHz sideband, or when installing the RAMS on the 6MHz.

Figure 1 shows in red the normal situation, in blue with the frequency to amplitude noise coupling divided by ~2 by adding the PRCL offset and in purple by adding the PRCL offset which zeros the coupling. The power fluctuation on B2 at ~1.5Hz improve by a factor 10 when reducing the coupling, on B4 there is also an improvement by a factor ~3, while on B4 12MHz the situation becomes worse, with fluctuations increasing by a factor 2. That would make sense if we improve the resonance condition for the carrier and degrade for the 6MHz sideband.

Figure 2 are the same times but with colors not in the same order. It includes also the B7 and B8 powers, and while on B8 the improvement is monotonic when improving the frequency to amplitude noise conversion, on B7 it is not the case. Is that a consequence of one arm being 3cm shorter than the other?

Lets assume that the PRC length is wrong by ~3mm, there are several options to resolve this:

1. Move PR by 3mm, which then requires retuning the input beam mode matching as it will change the input beam mode matching by one or a few percent
2. Change the sideband frequency by ~2kHz, but that would require moving the IMC end mirror by ~5cm to follow, so it is not a viable option
3. Move the input mirror by 3mm and maybe  also the end mirrors to keep the arm length the same. It might be the easiest to do and is the most reversible. It will also change the SRC length, but we have no measurements whether the SRC length is right or wrong, and it should be less critical because of the low finesse of the SRC.

Glitches might have stopped because the WI etalon loop error signal has finally become negative. The loop is asking for cooling the WI tower, which means turning the heating belt off. This switch off didn't happen immediately, but with an on/off for 10 minutes, so it is most likely due to the loop itself and not from an intentional switch off.

I have run a cross-correlation analysis using deltas for each glitch time, and all the _mean and _rms channels from the trend frame. I confirm what was highlighted by Bas in the previous comment:

1) Since April 26 20:00 UTC, the LSC_WI_HB_moni channel has started behaving oddly, with a sequence of square waves from zero to the set point. This is simultaneous with the appearance of the new glitches. The channel monitoring the electric potential, LSC_WI_HB_cmd_100Hz_FS, doesn't present similar behaviour. Figure 1 shows the glitchgram, with the new glitches showing up at the end of Apr 26, and the time series of the two channels monitoring the WI etalon.

2) The glitches in hrec are synchronous with the steps in the square wave visible in LSC_WI_HB_moni.

3) The correlation analysis with the _rms and the _mean trend channels has produced no other correlated channel with these glitches.

Glitches disappeared around 6:45 UTC. Has some action been taken? Indeed, square waves were seen also in the CEB UPS CURR T monitor, as for the old problem with WI heating belts that Bas pointed out.

These new glitches seem to originate from some noise source activating for ~2 minutes and off for ~4, similar to a square wave.

First of all, I identified the new glitch family selecting the omicron triggers with peak frequency lower than 40 Hz and SNR between 15 and 60. Then, plotting the temporal distances in the occurrences of consecutive glitches (similarly to what has been done for the 25-minute glitches), I noticed that these alternate between ~115 seconds and 250 or 225 seconds in the examined time interval: figure 1.

Figure 2 and 3 show the spectrograms of two consecutive glitches in hrec, and their whiten time series. The latter show opposite behaviours, with a spike up in one glitch, followed by a spike down in the next one.

Similar plots for the magnetometers at the West Input, show a step-like behaviour: figure 4 and 5.

The (non-whitened) time series of this magnetometer channel shows more clearly the square wave behaviour in the correspondence of each pair of glitches: figure 6.

The next task is finding what source is activating with a similar timing.

ITF found in LOW_NOISE_3_SQZ and in Science Mode.
At 18:30 UTC I set the ITF in CALIBRATION Mode to perform the planned measurements:

• 18:31 UTC - check hrec (CALIBRATED_DF_DAILY);
• From 18:46 UTC to 19:16 UTC - ISC Injections (inject_SSFS.py)

ITF back in Science mode from 19:16 UTC.

Guard Tour (UTC):
14:05 - 14:55 
15:57 - 16:32
19:06 - 19:36

DAQ
The process FCIM_MotSwitch stopped working at around 17:41 UTC. Process restarted via VPM at 17:44 UTC.

The ITF kept science mode all the shift.

Guard tours (UTC):
05:55 --> 06:30
08:00 --> 08:35
09:48 --> 10:33
11:55 --> 12:30

Nice find. Figure 1 and 2 made using omicron-plot show that these glitches in the WI MAG started at the same time as the glitches in h(t) around 20:00 UTC, and that they have not been happening the day before, and have been happening since then. There are magnetic glitches between 150Hz and 200Hz that have stopped a few hours before the glitches at ~30Hz started. The two might be related.

Figure 3 shows that this glitches are not visible on the NI MAG. So it is likely that the issue is close to the WI.

Next steps should be:

• Use the veto based on the WI MAG channel between 10Hz and 40Hz for GW analysis
• Try to find the source of the WI MAG glitches. Is there a piece of hardware that is starting to fail near WI

UPV + VetoPerf point to the following channels (without the _0 which is added in UPV to label the corresponding veto):

V0 → V1:ENV_WI_MAG_W_0, vetoed clusters: 454 (80.927 %)
V1 → V1:ENV_WI_MAG_N_0, vetoed clusters: 452 (80.570 %)
V2 → V1:ENV_WI_MAG_V_0, vetoed clusters: 452 (80.570 %)
V3 → V1:TCS_WI_CO2_ISSIN_0, vetoed clusters: 172 (30.660 %)
V4 → V1:ENV_CEB_ELECTRIC_0, vetoed clusters: 121 (21.569 %)

See https://scientists.virgo-gw.eu/DataAnalysis/DetCharDev/users/narnaud/UPV/20240428_glitches/V1:Hrec_hoft_16384Hz/perf/vp.html#V0 -> Click here to expand -> Time-frequency trigger distribution:    before/after veto.

In attachment the Omicron plots of h(t) and V1:ENV_WI_MAG_W, for comparison.

ITF found in LOW_NOISE_3_SQZ and in Science Mode. 
From 13:07 UTC to 13:08 UTC I set the ITF in ADJUSTING Mode to allow Nardecchia to turn the WI waveplate (see report #64116).
At around 17:50 UTC I was contacted by the LIGO Livingston Control Room, which reported that both LIGO ITF would postpone the planned calibrations by 24 hours. In agreement with the Commissioning coordinator, the planned Hrec calibration and LSC injections are postponed until tomorrow afternoon. 
ITF left locked.

Guard Tour (UTC)
13:01 - 13:29
15:01 - 15:29
17:50 - 18:20
19:41 - 20:20

A new noise is seen in the external magnetometer starting April 22 19:56 UTC. Figures 1 and 2. The noise has been detected and studied by Renato and Federico. Here is a report of the investigation so far.

The noise is impulsive, as seen in Figures 3,4,5. Observed in the time domain it consists of a carrier at about 8Hz switching on-off with a cycle of approximately 4.4s (0.23 Hz): on for 2.2s and off for 2.2s.

The noise is seen by all magnetometers site-wide. Internal magnetometers sees it a factor up to about 10 larger than the external "N" magnetometer, in Figure 6 and Figure 7 (the EXT_N magnetometer is the gray spectrum in all 3 plots). Magnetometers that see it a bit more intense are those close to NI, WI and WE towers.

A close look to the narrow spectral structure of the noise shows that it does not follow the mains, in Figure 8.

These hints suggest a DC-powered PWM source (Federico).

Bruco runs using ENV_NI_MAG_V as target, and EXCLUDING all magnetometers (ENV_*MAG*) channels, show that this disturbance is not coherent with any other ENV channel (https://scientists.virgo-gw.eu/DataAnalysis/DetCharDev/bruco/users/fiori/ENV_NI_MAG_V_1397853318_ENV/) with the exception of a small coherence with the ENV_CEB_ELECTRIC. In particular no coherence, nor evidence of the noise is seen in all IPS and UPS probes. No coherence is also measured with any other Virgo signal (https://scientists.virgo-gw.eu/DataAnalysis/DetCharDev/bruco/users/fiori/ENV_NI_MAG_V_1397853318/). While s is a good news for the interferometer, it does not help much locating the source.

Measurements will be done also to check for possible external sources, like it was for galvanic currents in the natural gas pipelines (https://logbook.virgo-gw.eu/virgo/?r=55542).

Nicola called me because the WI CO2 laser changed its state (see figure 1).
To address the change in the WI outer power, I first rotated the WI outer ring waveplate to recover the nominal  power to be injected into the ITF (11.59 UTC-12.00.26 UTC ) (ITF in ‘adjusting’ mode). Then, I adjusted the WI chiller set point to return the laser to its usual operating point, in particular I decreased it by 0.01 degrees, from 19.02 to 19.01 (12.21.04 UTC). The BCK chiller set point has been changed accordingly. This action did not solve the problem; in fact, the direction turned out to be wrong.
Thus, at 12.41 UTC, I increased the chiller set point from 19.01 to 19.03 degrees.
Finally, the standard situation has been restored (see fig.2). 

Upon arrival I found ITF in Science Mode (LN3_SQZ).
At 09:12 UTC ITF unlocked due to an earthquake in Japan with Mag 6.5 (fig2). L1, H1 already down.
ITF relocked, Science Mode back at 10:16 UTC.

ISC
The DMS reports a mistuning of B4_56MHz (LSC_SSFS_Err_DPHI) since few days. To be check. Plot (1). ISC expert present onsite informed.

TCS
After the relock the DMS notified a Power_DAS_OUT_is_out_of_range for WI CO2 and Beam_is_inside_ITF for CH PICKOFF (plot3). I alerted TCS OnCall.
At 11:59 and at 12:57 UTC I set in ADJUSTING for two minutes in order to allow Nardecchia to turn the WI plate and the WI chiller temperature setpoint.

Guard Tour (UTC)
05:03 - 05:29
07:01 - 07:27
10:56 - 11:24
 

As observed by Federico, the metronix magnetometer in MCB is having an issue starting from 2023 October 2 UTC 00:03:03 (Figure 1). The signal looks corrupted (Figure 2).  This does not seem an issue with the ADC and neither with the driving electronics, since other channels sharing the same boards are fine. We will investigate further at next maintenance.

This fauty condition was not signalled by the DMS DeadChannel since the associated rms was smaller than the set threshold. The rms threshold has been updated (0.35 to 1 nT) and this channel temporary shelved.